Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling

Visual storytelling is a task of generating relevant and interesting stories for given image sequences. In this work we aim at increasing the diversity generated while preserving informative content from images. We propose to foster informativeness story by using concept selection module that suggests set candidates. Then, utilize large scale pre-trained model convert concepts images into full stories. To enrich candidate concepts, commonsense knowledge graph created each sequence which candidates are proposed. obtain appropriate graph, two novel modules consider correlation among image-concept correlation. Extensive automatic human evaluation results demonstrate our can produce reasonable concepts. This enables outperform previous models margin on story, retaining relevance sequence.